Assessing Progress of Learning in Complex Domains

The specific problem addressed in this paper is the lack of assessment and feedback tools appropriate to support learning and instruction in complex and dynamic task domains. In such domains, representative problems can be identified but they often lack single solutions, and there is a great deal of variability between and among problems. As a consequence, in order to determine progress of learning and the efficacy of interventions, one must consider not only the solutions developed by learners but what they were thinking about when developing those solutions. Previously, think-aloud protocol analysis was used for the latter (Ericsson, 2001), but this methodology is researcher intensive and does not scale up for use in instructional settings. What are needed are tools that can be used in a variety of instructional settings, including classrooms, online settings, and informal learning environments. A review of what has been done in the past will be presented along with the most promising technologies. A specific approach which involves capturing a person’s conceptualization of a complex problem-solving situation and comparing that with an expert representation will be presented as a promising instructional and research solution to the general problem of assessing learning in complex domains.